In recent years, models based on the Transformer architecture have seen widespread applications and have become one of the core tools in the field of deep learning. Numerous successful techniques, such as parameter-efficient fine-tuning and efficient scaling, have been proposed surrounding their applications to further enhance performance. However, the success of these strategies has always lacked the support of rigorous mathematical theory. To study the underlying mechanisms behind Transformers and related techniques, we first propose a Transformer learning framework motivated by distribution regression, with distributions being inputs, connect a two-stage sampling process with natural language processing, and present a mathematical formulation of the attention mechanism called attention operator. We demonstrate that by the attention operator, Transformers can compress distributions into function representations without loss of information. Moreover, with the advantages of our novel attention operator, Transformers exhibit a stronger capability to learn functionals with more complex structures than convolutional neural networks and fully connected networks. Finally, we obtain a generalization bound within the distribution regression framework. Through the aforementioned theoretical results, we further discuss some successful techniques emerging with large language models (LLMs), such as prompt tuning, parameter-efficient fine-tuning, and efficient scaling. We also provide theoretical insights behind these techniques within our novel analysis framework.

Data attribution methods, which quantify the influence of individual training data points on a machine learning model, have gained increasing popularity in data-centric applications in modern AI. Despite a recent surge of new methods developed in this space, the impact of hyperparameter tuning in these methods remains under-explored. In this work, we present the first large-scale empirical study to understand the hyperparameter sensitivity of common data attribution methods. Our results show that most methods are indeed sensitive to certain key hyperparameters. However, unlike typical machine learning algorithms---whose hyperparameters can be tuned using computationally-cheap validation metrics---evaluating data attribution performance often requires retraining models on subsets of training data, making such metrics prohibitively costly for hyperparameter tuning.

Anomaly and failure detection methods are crucial in identifying deviations from normal system operational conditions, which allows for actions to be taken in advance, usually preventing more serious damages. Long-lasting deviations indicate failures, while sudden, isolated changes in the data indicate anomalies. However, in many practical applications, changes in the data do not always represent abnormal system states. Such changes may be recognized incorrectly as failures, while being a normal evolution of the system, e.g. referring to characteristics of starting the processing of a new product, i.e. realizing a domain shift. Therefore, distinguishing between failures and such ''healthy'' changes in data distribution is critical to ensure the practical robustness of the system. In this paper, we propose a method that not only detects changes in the data distribution and anomalies but also allows us to distinguish between failures and normal domain shifts inherent to a given process. The proposed method consists of a modified Page-Hinkley changepoint detector for identification of the domain shift and possible failures and supervised domain-adaptation-based algorithms for fast, online anomaly detection. These two are coupled with an explainable artificial intelligence (XAI) component that aims at helping the human operator to finally differentiate between domain shifts and failures. The method is illustrated by an experiment on a data stream from the steel factory.

Standard sequential decision-making paradigms aim to maximize the cumulative reward when interacting with the unknown environment., i.e., maximize $\sum_{h = 1}^H r_h$ where $H$ is the planning horizon. However, this paradigm fails to model important practical applications, e.g., safe control that aims to maximize the lowest reward, i.e., maximize $\min_{h= 1}^H r_h$. In this paper, based on techniques in sketching algorithms, we propose a novel planning algorithm in deterministic systems which deals with a large class of objective functions of the form $f(r_1, r_2, ... r_H)$ that are of interest to practical applications. We show that efficient planning is possible if $f$ is symmetric under permutation of coordinates and satisfies certain technical conditions. Complementing our algorithm, we further prove that removing any of the conditions will make the problem intractable in the worst case and thus demonstrate the necessity of our conditions.

Inverse halftoning is a technique used to recover realistic images from ancient prints (\textit{e.g.}, photographs, newspapers, books). The rise of deep learning has led to the gradual incorporation of neural network designs into inverse halftoning methods. Most of existing inverse halftoning approaches adopt the U-net architecture, which uses an encoder to encode halftone prints, followed by a decoder for image reconstruction. However, the mainstream supervised learning paradigm with element-wise regression commonly adopted in U-net based methods has poor generalization ability in practical applications. Specifically, when there is a large gap between the dithering patterns of the training and test halftones, the reconstructed continuous-tone images have obvious artifacts.

We thank the reviewers for all of these valuable comments. We provide point-by-point responses below. Re: generalize to other applications. Cooperative Navigation problem (Lowe et al. '17) and achieved a competitive mean reward (-4.8) against MADDPG Specifically, the stochastic target selection will make the executor inefficient to learn. We will further discuss the factors of each component in the next revision.

We thank the reviewers for their constructive and positive comments. They will improve the quality of the paper. As an instance in RL, we mention the problem of "active exploration in MDPs" (see [28]), where the Reiterating the discussion in Section 2.3, let us consider the small-budget regime, and We will provide a footnote in page 7 to clarify this. This is indeed a nice remark. As a result, the theorem is valid even if irreducibility and aperiodicity are dropped.

We kindly thank the reviewers for their detailed reviews, valuable feedback and suggestions for improvement. Indeed, our proof of the new SW theorem relies on an "ordering" of the coordinates of arbitrary equivariant SW theorem under arbitrary finite group action would be desirable, however the proof is out of our reach as of today. In a way, this limitation is similar to the distinction between "point clouds" (which in We will add this discussion in the paper, and mention it in the abstract. In its "deep" original version, it covers all type of "Message-Passing" GNNs, but not spectral GNNs which use powers of the adjacency matrix. We will clarify this in the final version.

--Special tags such as AprilT ags that facilitate image processing and pattern recognition are useful in practical applications. In close and private environments, identity security is unlikely to be an issue because all involved AprilT ags can be completely regulated. However, in open and public environments, identity security is no longer an issue that can be neglected. T o handle potential harm caused by adversarial attacks, this note advocates utilization of adjustable AprilT ags instead of fixed ones. Special tags that facilitate image processing and pattern recognition are useful in practical applications.

MSC 804 5 - 0020 - 10 St. Louis, MO 63110 Financial Support: P30 - AR073752 Conflict of interest: No conflicts declared. Page 2 of 13 Roberson 2025 - LLMs in medicine Abstract The medical ecosystem consists of the training of new clinicians and researchers, the practice of clinical medicine, and areas of adjacen t research. There are many aspects of these domains that could benefit from the application of task automation and programmatic assistance. Machine learning and artificial intelligence techniques, including large language models ( LLMs), have been promised to deliver on healthcare innovation, improving care speed and accuracy, and reducing the burden on staff for manual interventions . However, LLMs have no understanding of objective truth that is based in reality. They also represent real risks to the disclo sure of protected information when used by clinicians and researchers. The use of AI in medicine in general, and the deployment of LLMs in particular, therefore requires careful consideration and thoughtful application to reap the benefits of these technol ogies while avoiding the dangers in each context. Page 3 of 13 Roberson 2025 - LLMs in medicine Abbreviations AI: Artificial intelligence CSP: Constrained solution problems EHR: Electronic health records GPT: Generative pre - trained transformer LLM: Large language model ML: Machine learning RAG: Retrieval - augmented generation Page 4 of 13 Roberson 2025 - LLMs in medicine What is a large language model? Large language models are one of the most hyped artificial intelligence technologies of the past few years. Buzz terms associated with them include artificial intelligence, machine learning, a nd deep learning.